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ABSTRACT 

The  design  section  is  responsible  for  designing 
projects  for  completion  by  contract,  overseeing  designs 
done  by  architect-engineer  firms  or  the  Army  Corps  of 
Engineers  (COE) .  The  impact  a  design  section  has  on  an  Air 
Force  base  is  enormous.  The  programmed  budget  for 
construction  for  1987  was  over  *3  billion,  or  about  15 
percent  of  the  Air  Force  operations  and  maintenance  budget. 

Performance  is  important  in  any  organization  with  this 
large  of  an  impact.  That  performance  should  constantly  be 
improved.  Before  performance  can  be  improved  it  must  be 
accurately  measured.  The  literature  available  suggests 
that  to  accurately  measure  the  performance  of  engineers, 
one  must  first  identify  key  dimensions  of  performance 
which,  when  accomplished,  will  assure  effectiveness.  After 
effectiveness  has  been  assured,  then  efficiency, 
productivity  and  quality  need  to  be  measured.  Together 
these  four  things  give  a  picture  of  performance. 

A  case  study  of  an  Air  Force  Civil  Engineering  design 
section  was  done  with  the  purpose  of  documenting  the  system 
used  for  performance  measurement.  The  system  used  measured 
effectiveness,  performance  against  schedule, 
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efficiency  and  productivity.  Some  capability  of  assuring 
quality  was  also  built  into  the  management  system. 

The  system  that  is  used  by  the  section  under  study  does 
provide  a  limited  capability  to  measure  performance. 
However,  this  system  uses  mostly  subjective  measures  and 
little  actual  quantifiable  data.  This  is  a  weakness  if  fine 
differences  in  performance  need  to  be  measured,  as  would  be 
the  case  for  a  performance  improvement  system. 
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DESIQV  AMD  IMPLEMEHTATIOV 
OF  A  PEBFOHMAMCE  MEASUBEMEVT 
SYSTEM  IV  THE  CIVIL  EVQIHEEBIVG 
DESIGV  SECTIOV:  A  CASE  STUDY 


I .  Introduction 


Background 

"The  mission  of  Air  Force  Civil  Engineering  (AFCE)  is 
to  provide  the  necessary  assets  and  skilled  personnel  to 
prepare  and  sustain  global  installations  as  stationary 
platforms  for  the  projection  of  aerospace  power  in  peace 
and  war’  (8:IMC  83-2).  The  design  section  fits  into  the 
mission  by  designing  projects  for  completion  by  contract, 
and  overseeing  designs  done  by  architect-engineer  (AE) 
firms  or  the  Army  Corps  of  Engineers  (COE).  In  addition  to 
design,  there  are  additional  duties  as  assigned.  These 
duties  can  be  best  characterized  as  serving  the  function  of 
staff  engineers,  for  not  only  AFCE,  but  for  the  rest  of  the 
chain  of  command  as  well.  The  design  section  has  a 
multitude  of  studies,  reports,  suggestions  and  programs  to 
deal  with  at  any  one  time. 

'The  base  civil  engineer  manages  or  otherwise 
administers  some  40  to  60  percent  of  most  instal lations 
operations  and  maintenance  (0&M)  budget’  (17:53).  In 


addition,  approximately  15  percent  of  the  0&M  budget  for 
the  Air  Force  goes  through  AFCE  in  the  form  of  completed 


designs  ready  for  contracting  (35:ch  A).  The  impact  the 
design  section  has  on  the  base  is  enormous.  This  impact  can 
be  either  positive  or  negative.  Each  project  completed 
affects  both  the  work  environment  and  the  ability  to 
perform  the  mission.  Likewise,  each  needed  project  left  out 
will  hinder  the  accomplishment  of  the  mission.  The  rest  of 
AFCE  also  benefits  from  good  designs,  in  that  required 
maintenance  is  reduced. 

Research  Problem  Statement 

A  performance  measurement  model  needs  to  be  developed 
and  tested  for  the  engineering  design  section.  Although 
several  attempts  have  been  made,  the  results  to  date  have 
not  been  totally  successful.  One  way  of  approaching  the 
problem  of  measuring  performance  in  design  would  be  to  find 
a  design  section  that  had  developed  and  used  a  performance 
measurement  system  on  their  own. 

One  such  design  section  exists  at  Tinker  Air  Force 
Base  (AFB) ,  Oklahoma.  This  thesis  investigated  and 
evaluated  the  system  of  performance  measurement  that  Tinker 
AFB’s  design  section  has  been  using. 

Ob j  ectives 

Primary  Obj active .  The  primary  objective  of  this 
study  was  to  investigate  and  document  the  performance 
measurement  system  in  use  in  the  design  section  at  Tinker 
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AFB.  This  included  documenting  the  development  and 
implementation  of  the  system  as  well  as  how  it  has  actually 
been  used.  Part  of  this  documentation  included  an 
investigation  of  the  validity  of  the  system.  Finally,  the 
system’s  effectiveness  in  improving  performance  was 
investigated . 

Secondary  Objective.  The  second  objective  was  to 
compare  the  performance  measurement  system  used  in  Tinker 
AFB ’ s  design  section  to  the  recommendations  of  the  experts 
as  presented  in  the  literature  review. 

Scope 

This  study  focused  on  documentation  of  the  performance 
measurement  system  as  used  by  the  design  section  at  Tinker 
AFB.  This  required  an  investigation  into  the 
recommendations  of  the  experts  about  how  a  performance 
measurement  system  should  be  designed  and  used.  However, 
this  is  not  to  say  that  a  better  system  or  an  alternative 
proposal  should  be  developed  from  the  literature  review. 

The  literature  review  is  only  intended  to  provide  a  basis 
of  comparison  for  the  system  to  be  Studied. 

Limitations 

There  are  two  sources  of  limitation  that  enter  this 
thesis.  First,  the  performance  measurement  system  being 
studied  was  designed  by,  and  for  the  use  of,  the  management 
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of  the  design  section  of  Tinker  AFB .  Therefore,  it  may  not 
fit  the  suggestions  of  the  experts  exactly.  This  was  one 
of  the  areas  of  investigation. 

Second,  because  this  is  a  case  study,  the  knowledge 
gained  may  not  universally  fit  other  design  sections.  The 
peculiarities  of  Tinker  AFB’s  design  section  may  make  the 
model  unusable  or  only  partially  usable  by  another  design 
section. 

Methodology 

This  thesis  was  performed  in  three  parts:  literature 
review,  case  study,  and  analysis.  In  the  literature  review 
basic  background  was  gained.  From  this  background  a  model 
for  comparison  was  developed.  This  included  not  only  the 
basic  requirements  of  a  measurement  system  but  the 
development  and  implementation  of  a  system,  and  specific 
work  related  to  the  AFCE  design  section  that  had  been  done 
by  past  researchers.  This  model  became  a  point  for 
comparison . 

The  second  step  was  to  actually  perform  the  case 
study.  A  case  study  was  used  because,  although  several 
studies  have  focused  on  what  could  be  done  in  AFCE,  no 
studies  have  been  done  about  what  is  presently  being  used 
in  AFCE.  This  left  a  gap  between  the  academic  research  and 
the  existing  systems  used  by  design  sections.  One  of  the 
benefits  of  a  case  study  is  that  it  allows  for  more 
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flexibility  in  data  collection.  This  allows  the  researcher 
to  mold  the  study  to  fit  unexpected  conditions  that  may  be 
encountered  in  the  field.  This  flexibility  allowed  the 
research  to  readily  explore  the  existing  measurement 
system.  This  was  necessary  because  of  the  small  amount  of 
information  available  about  the  existing  system.  This  made 
the  use  of  any  other  research  tool  impractical. 

Another  good  reason  for  the  use  of  a  case  study  is  to 
document  information  that  is  unique  or  not  previously 
available  to  researchers  (37:18).  The  data  under 
investigation  fits  this  last  category  because  it  deals 
specifically  with  what  exists  in  AFCE.  As  previously 
stated,  several  studies  have  been  done  about  what  was 
available  for  use,  and  several  models  have  been  developed. 
However,  not  much  study  has  been  done  about  what  is  in  use. 
This  study  provides  that  unique  information  by  documenting 
a  system  that  is  in  use  in  AFCE.  Another  purpose  of  a  case 
study  is  to  generate  hypotheses.  This  would  be  helpful  in 
bridging  the  gap  between  what  has  been  done  academically 
and  what  exists  in  AFCE  (37:18). 

The  case  study  focuses  on  three  areas.  First,  the 
reason  the  measurement  system  was  developed,  and  how  it  was 
developed,  were  determined.  Second,  the  system  itself  was 
documented.  This  documentation  focuses  on  the  system 
organization  and  development  and  includes  how  it  is 
actually  used.  Finally,  this  study  determined  how  well  the 
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system  has  been  able  to  meet  the  needs  of  the  management  of 
the  organization. 

The  final  step  in  this  thesis  was  to  compare  the 
features  of  successful  performance  measurement  models 
discussed  in  the  literature  review  with  the  system  used  at 
Tinker  AFB ,  as  documented  in  the  case  study.  This 
comparison  focuses  on  the  validity  of  the  system  under 
investigation.  This  includes  technical  validity, 
organizational  validity  and  use.  Technical  validity  refers 
to  'measuring  what  the  model  purports  to  measure*  (26:153). 
Bumsey  (24:33)  distinguishes  between  technical  validity  of 
the  model  and  technical  validity  of  the  organizational 
model,  which  'is  a  function  of  such  concepts  as  ease  of 
measurement,  selection  of  appropriate  variables  and 
reliability/validity  of  the  underlying  variables'  (24:33). 

’  Organizational  validity  refers  to  the  acceptability  of 
the  model  to  organizational  users*  (26:153).  The  final  test 
was  how  well  the  model  was  used  to  improve  performance. 

To  summarize,  this  thesis  investigated  the 
performance  measurement  system  in  use  at  Tinker  AFB  and 
compared  it  to  the  recommendations  of  the  literature 
available.  The  comparison  focused  specifically  on  the 
validity  of  the  system.  Recommendations  were  made  about  how 
the  validity  and  usefulness  of  the  system  can  be  improved. 
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Juati f icatlon 


There  are  at  least  three  good  reasons  to  test  the 
model  In  use  at  Tinker  AFB .  The  first  reason  is  the  need  to 
improve  performance.  This  need  drives  us  to  measure 
performance.  The  second  reason  is  that  AFCE  has  no  proven 
successful  model  of  performance  for  the  design  section. 

The  only  way  to  get  a  successful  model  is  to  keep  trying. 
Since  Tinker  AFB  already  has  a  model,  it  only  stands  to 
reason  that  it  should  be  evaluated.  This  will  also  provide 
future  researchers  information  which  might  allow  them  to 
bridge  the  gap  between  what  was  done  at  Tinker  AFB  and  what 
is  recommended  by  the  literature. 

The  last  reason  to  test  this  model  is  that  it  is  part 
of  a  design  schedule  program  developed  at  Tinker  AFB  for 
the  Wang  computer  and  the  work  information  management 
(WIMs)  software.  This  computer  system  is  in  the  process  of 
being  installed  at  every  major  Air  Force  base  in  the 
continental  United  States.  This  design  schedule  program  is 
already  in  the  process  of  being  distributed  and  implemented 
at  other  bases,  because  developing  a  design  schedule  seems 
to  have  been  a  high  priority  and  a  difficult  task  for  AFCE 
design  sections  for  many  years.  If  this  program  gains 
acceptance  and  recognition  at  those  other  bases  it  could  be 
implemented  throughout  AFCE  as  part  of  the  WIMs  software 
for  the  Wang  computer  system.  If  the  performance 
measurement  part  of  this  program  is  good,  it  should  be 
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implemented  at  other  bases.  If  the  performance  measurement 
part  of  this  program  is  technically  incorrect,  it  should  be 
remedied . 

The  possibility  exists  that  this  research  could  bring 
forth  a  usable  system  of  performance  measurement  for  the 
design  section.  Such  a  system  could  greatly  benefit  Air 
Force  Civil  Engineering. 
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I I .  Li terature  Review 


Introduction 

The  literature  review  was  intended  to  explore,  develop 
and  explain  what  a  performance  improvement  program  la  and 
can  be.  This  review  atarts  on  a  very  broad  base ,  explaining 
the  importance  of  performance  and  productivity.  The  next 
atep  is  to  present  the  basics  of  a  performance  improvement 
program.  This  explanation  will  include  the  need  for 
improvement,  definitions,  program  effectiveness,  source  of 
improvements,  measurement  system  development  and  design, and 
val idity . 

Finally  the  literature  review  explains  specific 
aspects  of  public  sector.  Air  Force,  white  collar,  and 
engineering  productivity.  This  effectively  breaks  the 
literature  review  into  two  parts:  mechanics  and  people. 

Both  of  these  aspects  are  presented  starting  with  the  broad 
case  and  moving  to  the  more  narrow  focus. 

Meed  for  Improvement 

"The  failure  of  U.S.  productivity  to  keep  up  with  the 
Japanese  and  the  rest  of  the  world’s  major  industrialized 
nations  is  a  growing  national  concern.  Our  national  leaders 
have  stated  that  the  economic  survival  of  our  factories, 


0 


our  standard  of  living,  and  ultimately  the  survival  of  our 
economic,  political,  and  governmental  systems  could  be  at 
stake'  (10:653).  Others  differ  slightly,  saying,  'The 
United  States  has  been,  and  continues  to  be,  the  most 
productive  nation  in  the  world'  (28:5).  However,  there  are 
nations  that  are  gaining  fast.  This  gain  by  other  nations 
translates  to  a  relative  decrease  in  our  standard  of 
living,  and  an  increase  in  the  gaining  nations’  standards 
(28:5).  Either  way,  the  experts  agree  that  our  nation’s 
standard  of  living  is  at  stake. 

The  Federal  Government  is  concerned  about  its 
performance  also.  This  stems  not  only  from  the  national 
interest  in  productivity,  but  also  from  budgetary 
constraints.  ‘At  a  time  when  we’re  looking  for  ways  to 
reduce  the  government  spending  and  curb  the  deficit,  it 
only  makes  sense  to  encourage  productivity  which  will  save 
the  taxpayer's  money  by  making  the  government  more 
efficient'  (18:146).  This  was  reflected  when  the  President 
issued  Executive  Order  12552  on  25  February  1986.  This 
order  mandates  a  20  percent  increase  in  productivity  by 
1992.  'The  program  has  the  twin  goals  of  achieving 
substantial  improvements  in  government  efficiency  and  of 
changing  the  way  managers  manage--and  thus  makes  it  clear 
that  managers  must  engage  in  systematic  and  continuing 
efforts  to  Improve  productivi ty '  (9:252). 


The  need  fop  improving  the  performance  of  the  deaign 
section  of  Air  Force  Civil  engineering  (AFCE)  is  driven  by 
at  least  two  other  major  concerns  to  the  United  States  Air 
Force.  The  first  concern  is  centered  around  the  mission  of 
the  design  section,  which  is  to  design  projects  for  new 
construction  and  renovation,  for  completion  by  contract  or 
in  house  forces.  The  design  section  is  constantly  being 
tasked  to  do  more,  yet  the  design  budget  often  does  not 
increase  enough  to  cover  the  task.  By  improving  the 
performance  of  the  design  section,  the  mission  of 
completing  more  projects  on  a  restricted  budget  will  be  met 
more  effectively.  The  demand  for  performance  comes  in  a 
time  of  real  budget  decrease,  while  confronted  with 
increased  need  (33:a-21) .  The  design  section’s  primary 
objective  is  to  accomplish  more  with  less. 

The  second  concern  is  the  national  view  that  the 
federal  government  is  not  able  to  manage  efficiently. 
Although  little  can  be  done  to  convince  critics,  a  valid 
performance  measurement  system  could  be  helpful  in  swaying 
the  public’s  views,  if  it  could  ’measurably  demonstrate 
that  the  government  is  being  efficiently  managed*  (9:253). 

In  the  November  1983  QAO  (General 
Accounting  Office)  report  entitled  Increased  Use 
of  Productivity  Management  Can  Help  Control 
Government  Costs .  and  in  a  subsequent  article 
published  in  the  spring  1984  National 
Productivity  Review  ,  Peter  Lemonias  and  Brian 
Usilaner  argue  convincingly  that  seven  key 
elements,  culled  from  both  public-  and  private 


sector  practices  are  essential  for  an  effective 
productivity  effort.  These  elements  are: 

1.  A  manager  serving  as  a  focal  point  for 
productivity  in  the  organization; 

2.  Top  level  support  and  commitment; 

3.  Written  productivity  objectives  and 
goals  and  an  organization  wide  productivity  plan; 

4.  Productivity  measures  that  are 
meaningful  to  the  organization; 

5.  Use  of  the  productivity  plan  and 
measurement  system  to  hold  managers  accountable; 

6.  Awareness  throughout  the  organization  of 
productivity’s  importance  and  involvement  of  the 
employees  in  the  productivity  effort;  and 

7.  An  ongoing  activity  to  regularly 
identify  productivity  problems  and  opportunities 
for  Improvement  throughout  the  organization 
(9:234). 

Buried  in  the  middle  of  the  above  list  is  one  of 
the  things  that  makes  performance  improvement  systems 
different  from  what  managers  did  in  the  past:  the 
performance  measures.  Without  a  performance  measurement 
system,  there  is  little  or  no  way  to  tell  how  some 
management  decision  affected  performance.  After  all,  ‘you 
cannot  manage  what  you  cannot  measure*  (12:17). 
‘Productivity  measurement  is  an  essential  element  of  an 
effective  productivity  improvement  effort.  The  measures 
need  not  be  precise,  total  factor  measures.  Often,  a  series 
of  measures  that  are  easy  to  understand  and  calculate  and 
that  are  meaningful  to  managers  and  employees  is  more 
useful*  (19:147).  Improved  performance  is  one  of  the  things 
that  managers  have  been  working  to  achieve  since  the 
beginning  of  scientific  management.  The  improvement  of 
performance  relies  on  old  and  proven  management  principles. 
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The  only  real  difference  is  that  now  instead  of  simply 
trying  to  manage ,  managers  are  becoming  interested  in 
directly  managing  performance.  Before  the  implementation 
of  productivity  programs,  productivity  was  an  outcome  of 
management,  but  the  quest  for  productivity  was  far  less 
organized . 

Definitions 

In  much  of  the  literature  the  word  productivity  and 
performance  are  not  well  defined.  For  instance,  the  terms 
performance  and  productivity  are  often  used 
interchangeably.  Performance  is  a  more  general  term  that 
does  not  carry  with  it  the  emotional  appeal  engendered  by 
the  term  productivity.  When  discussing  performance  one  must 
define  the  aspects  of  performance  that  are  deemed  most 
essential . 

The  general  aspects  of  performance  that  are  measured 
consist  of  quality,  effectiveness,  efficiency,  quality  of 
work  life,  innovation,  and  productivity.  Some  or  all  of 
these  things  may  be  important  to  the  survival  of  an 
organization  (28:41). 

Qual i ty .  Quality  is  the  degree  to  which  the  products 
produced  conform  to  requirements ,  speci f icati ons ,  or 
expectations.  Customer  satisfaction  is  inherent  in  the  idea 
of  quality.  Another  aspect  of  quality  is  product 
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performance;  the  product  must  meet  the  need  for  which  it 


was  produced . 

Ef f ectiveness .  Effectiveness  is  the  degree  to  which 
the  organization  accomplishes  what  it  set  out  to 
accomplish.  Measurement  of  effectiveness  often  relies  on 
other  aspects  of  performance,  such  as  quality,  quantity  and 
timeliness . 

Ef f iciencv .  Efficiency  is  the  ratio  of  the  resources 
expected  to  be  consumed  to  the  resources  actually  consumed. 
This  definition  of  efficiency  rests  on  estimates  or 
expectations  which  form  the  baseline  for  comparison. 

Quality  of  work  life.  Quality  of  work  life  is  the  way 
employees  respond  to  sociotechnical  aspects  of  the 
organization.  This  is  reflected  in  absenteeism,  employee 
turnover,  and  other  employee  behaviors. 

Innovation .  Innovation  is  applied  creativity. 
Innovation  is  how  well  the  organization  develops  and 
implements  new  and  better  ideas,  as  measured  in  the 
relative  features  of  the  products  produced. 

Productivity .  Productivity  is  the  ratio  of  the 
quantities  of  output  to  the  quantities  of  input. 
Productivity  is  different  from  efficiency  in  that  it  is  not 
based  on  any  estimates. 


Program  Ef f ectivenesa 

How  effective  are  performance  management  systems? 

Case  studies  are  available  of  public  sector  and  private 
sector  systems.  These  studies  show  that  each  organization 
is  different,  and  the  improvements  that  are  possible  are 
also  different.  According  to  Lemonias  and  Usilaner,  average 
performance  improvements  of  about  nine  percent  annually 
have  been  realized  by  the  companies  they  interviewed,  with 
state  and  local  government  improvements  ranging  from  two  to 
five  percent  (10:139). 

In  Calgary,  Canada,  the  city  engineers  found 
incredible  savings.  "In  the  first  year  they  had  a  * 4 
million  surplus  against  an  *18.5  million  budget.  Of  this  *4 
million,  one  third  could  be  attributed  directly  to  improved 
productivity  and  lower  unit  costs  ..." (27 : 142) . 

The  U.S.  Copyright  Office  has  implemented  a 
productivity  improvement  system  with  excellent  results. 

Over  the  last  five  years  they  have  had  ‘a  23  percent 
increase  in  work  load  along  with  a  19  percent  decrease  in 
staff  while  achieving  a  15  percent  increase  in 
productivity*  (23:155). 

Stevens  found  that  in  an  analysis  of  city  government, 
'Appropriate  changes  in  management  practices  can  result  in 
savings  of  up  to  or  even  greater  than  50  percent’ 

(31:398) . 
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Sources  of  Improvement 


Where  does  the  added  performance  come  from?  It 
appears  that  performance  improvements  come  from  three 
areas:  mechanical,  organizational,  and  personnel. 

Mechanical  improvements  include  any  plant  or  equipment 
changes  that  cause  the  system  to  perform  better. 
Organizational  changes  are  those  changes  in  the 
organizational  structure  that  make  the  organization  better 
capable  of  supporting  the  workers  in  their  job  performance. 
Personnel  changes  include  a  multitude  of  things  such  as 
changes  in  work  methods,  rules,  regulations,  or  job 
assignment . 

The  sources  for  these  changes  can  come  from  the 
workers  and  a  more  participative  approach  to  management,  or 
from  one  of  a  multitude  of  formal  methods  of  analysis.  The 
participative  approach  usually  starts  when  an  organization 
begins  a  performance  improvement  system.  The  first  step  is 
often  to  implement  a  measurement  system.  This  requires 
management  to  evaluate  the  reasons  for  being  in  business. 
The  mission  usually  comes  into  focus.  A  multitude  of 
activities  are  identified  as  not  being  mission  essential. 

A  specific,  formal  method  of  identifying  what  is  and 
is  not  important  is  organizational  analysis  (36:171)  or 
organization  function  analysis  (OFA)  (5) .  Both  of  these 
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methods  ape  very  similar  OFA  will  be  outlined  here  because 
a  more  in-depth  presentation  is  available. 

The  first  step  in  the  OAF  process  is  to  secure  the 
total  commitment  of  upper  management  (5:ch  9).  For  this 
system  to  really  work  upper  management  must  demonstrate  a 
real  willingness  to  change,  to  challenge  traditional 
practices  and  lead  others  to  do  the  same.  Thi3  starts  with 
a  senior  management  training  program  that  usually  takes  a 
full  day.  Another  day  is  then  devoted  to  program  planning. 
This  planning  includes  objectives,  scope,  participation, 
schedule,  and  the  method  of  communication  to  be  used.  The 
program  should  be  as  broad  as  practical  and  designed  for 
long-term  change.  Short  term  perceptions  should  be 
eliminated,  because  they  will  limit  the  program  too  much. 

The  first  step  in  determining  the  scope  is  to 
determine  what  functions  should  be  involved.  After  the 
program  scope  is  determined,  the  OFA  team  can  be  selected. 
Volunteers  should  not  be  accepted.  Team  members  should  be 
solicited  from  the  functions  involved,  not  to  be 
representatives,  but  to  add  breadth  of  knowledge.  Different 
experience  levels  should  also  be  included:  the  very  new, 
and  the  very  experienced.  Team  members  should  be  respected 
by  management  and  their  own  peers.  The  last,  and  possibly 
most  important,  criteria  is  attitude.  The  members  selected 
must  have  a  constructive  attitude,  and  a  genuine  desire  to 


work  with  others  to  improve  productivity.  The  size  of  the 
team  will  vary,  but  for  a  typical  AFCE  squadron  of  about 
550  people,  a  team  of  about  20  members  should  be  selected. 

The  next  step  is  to  train  the  team.  This  should 
include  20  to  30  hours  of  formal  instruction  in  OFA.  Good 
training  is  absolutely  vital  to  the  success  of  any 
participative  management  program.  After  the  team  is 
trained,  a  plan  must  be  developed  for  information 
gathering.  This  plan  should  be  specific  enough  to  include 
who  will  actually  be  interviewed.  The  people  that  perform 
each  operation  in  the  work  place  should  be  interviewed.  The 
purpose  is  to  gather  information  about  what  actually  takes 
place,  rather  than  what  the  manual  or  the  supervisor  says 
is  supposed  to  happen. 

The  team  will  then  divide  the  interviews  down  into 
areas  of  work  similarity,  called  modules.  Then  the  OFA  team 
will  form  two-man  groups;  each  group  will  take  one  module. 
The  interviews  should  determine  quantitative  and 
qualitative  data  about  the  job.  The  quantitative  data 
should  include  demand  volumes,  hours  of  effort  involved  per 
month,  cycle  times,  and  expenses.  The  qualitative  data 
should  include  specific  observations,  suggestions  and 
recommendations . 

The  next  phase  is  information  analysis  (5:ch  10). 
During  this  phase  there  are  three  key  functions.  The  first 
function  is  to  maintain  the  master  list  of  definitions  of 


functions,  demands,  and  demand  volumes.  These  lists  are 
updated  constantly  as  data  is  analyzed. 

The  second  key  function  is  to  coordinate  and  maintain 
analysis  summary  sheets.  This  is  the  final  collection  of 
quantitative  knowledge.  First,  all  of  the  data  gathered  is 
transferred  onto  operation  summary  sheets.  These  sheets 
briefly  tell  who  does  what,  why  they  do  it,  how  often,  and 
how  long  it  takes.  The  operation  summary  sheets  then  are 
used  as  source  information  to  fill  out  the  function  summary 
sheets.  These  sheets  tally  all  of  the  information  about  any 
given  function.  The  last  step  is  to  fill  out  OFA  ratio 
summary  sheets.  These  sheets  give  all  of  the  pertinent  data 
about  each  given  function. 

The  third  part  of  OFA  is  taking  the  key  observations 
that  were  made  during  the  information  gathering  and 
processing  stages,  condensing  the  observations  into  one 
line  statements  and  sorting  them  by  function.  Finally  a 
function  flow  diagram  is  developed.  This  diagram  shows  how 
work  actually  flows  in  the  organization. 

The  next  step  is  evaluation  and  improvement 
identification  (5:ch  11).  In  this  step  the  team  tries  to 
evaluate  the  information  and  determine  what  it  really 
means.  The  team  then  develops  solutions  and  methods  of 
improvement.  This  is  done  by  starting  with  each  demand  the 
organization  faces,  and  then  each  function  that  answers 
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that  demand,  and  then  each  operation.  In  all  of  these 


steps,  the  same  type  of  questions  are  asked: 

1.  Is  the  demand  valid  and  what  effect  does  it  have? 

2.  Is  there  a  better  way  of  doing  it? 

3.  Can  we  reorganize  to  better  meet  these  needs? 

After  the  analysis  has  been  done,  the  recommendations 
are  gathered  into  a  report  and  forwarded  to  top  management. 
These  recommendations  are  then  modified  as  neceaaa.ry ,  and  a 
plan  for  implementation  is  developed. 

Some  other  methods  are  less  formal:  Bolte  recommended 
a  similar  approach  to  essential  task  identification.  He 
recommended  that  managers  ask  in  relation  to  each  activity, 
‘Should  this  task  be  done  at  all?'  (3:136).  If  the  answer 
isn’t  adequate,  the  activity  is  eliminated.  The  other  part 
of  this  process,  Bolte  suggests,  includes  getting  the  right 
person  to  accomplish  the  right  job.  In  addition,  areas 
that  need  other  improvements  usually  surface  at  this  time. 
Rules  need  changed.  Equipment  needs  replaced,  or 
purchased . 

In  addition  to  the  benefits  received  from  using 
improved  work  techniques,  the  individual  performance  of 
employees  may  improve.  A  sense  of  competition  may  promote 
this  improvement,  or  in  the  private  sector,  monetary 
incentives  are  often  used.  Another  possibility  that  was  not 
mentioned  by  any  of  the  authors,  is  the  ‘Hawthorne  effect* 
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(6:209),  in  which  workers  improve  performance  simply 
because  they  are  being  watched. 

Performance  improvement  comes  from  both  improved 
methods  of  work  and  improved  employee  performance.  Sheehy 
concluded  that  ‘the  potential  for  improvement  (in 
performance)  tends  to  increase  exponentially  the  further 
from  the  shop  floor  and  into  the  management  system  one 
probes ... ‘ (27 : 144)  .  There  are  great  possibilities  for 
improvement  and  the  possibilities  increase  the  further  up 
the  chain  of  command  they  are  implemented.  However,  before 
performance  improvement  can  begin,  commitment  to  improve 
performance  must  be  obtained  and  measurement  should  start. 

Types  of  Measurement  Systems 

Most  measurement  systems  claim  to,  or  say  that 
they,  measure  specific  areas  of  performance.  However, 
because  of  the  weak  definitions  incorporated,  the  systems 
often  include  many  performance  measures. 

Sink  describes  productivity  measures  as  ratios  of 
output  over  input.  He  further  breaks  these  ratios  into  two 
broad  categories  based  on  the  type  and  quantity  of  measures 
used.  The  first  category  is  whether  the  measure  is  static 
or  dynamic.  Static  productivity  ratios  are  simply  output 
divided  by  input  for  some  given  period  of  time.  Dynamic 
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measures  are  basically  static  measures  of  one  time  frame 
divided  by  static  measures  from  another  time  f rame ( 28 : 26) . 

There  are  three  types  of  productivity  measure  in  each 
of  the  above  categories;  partial - factor ,  multi-factor,  and 
total  factor.  Each  of  the  measures  is  a  ratio;  however,  the 
denominators  (inputs)  are  different.  In  partial-factor 
analysis,  only  one  input  is  used  in  the  ratio.  In  multi¬ 
factor  analysis  more  than  one  type  of  input  is  used  and  in 
total-factor  analysis  all  of  the  system  inputs  are  used 
(28:26)  . 

Another  important  area  is  that  of  surrogate  measures. 
Surrogate  measures  are  taken  from  areas  that  are  closely 
correlated  with  the  types  of  measurements  that  are  actually 
desired.  These  are  usually  used  when  direct  measurements 
are  unavailable.  Surrogate  measures  are  frequently  used  for 
white  collar  workers. 

Sink  recommends,  for  use  at  the  level  under  study 
here,  either  the  normative  performance/productivity 
measurement  method  (NP/PMM)  or  the  multi-criteria 
performance/  productivity  measurement  technique  (MCP/PMT) 
(29:282).  NP/PMM  is  mostly  a  method  of  developing 
performance/productivity  measures  using  a  structured  group 
process  such  as  the  nominal  group  technique  or  the  Delphi 
method.  This  assures  the  consensus  of  the  workers.  NP/PMM 
involves  5  stages. 
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Stage  one  involves  structured  group  process  generating 


a  group  of  prioritized  productivity  measures,  ratios  and 
indicators.  The  reason  for  using  a  participative  approach 
is  to  'ensure  adequate  motivation,  commitment,  and 
accountability  on  the  part  of  key  participants  for 
implementation  and  acceptance  of  the  resultant  productivity 
measurement  system'  (29:275). 

Stage  two  involves  having  a  productivity  analyst  take 
the  information  from  stage  one  and  determine  how  to 
implement  the  measures  suggested.  The  analyst  also 
determines  how  and  where  to  obtain  the  data.  After  the 
analyst  works  on  the  measures,  the  group  helps  shape  them 
into  a  usable  system. 

Stage  three  Involves  briefing  the  management  and 
workers  about  the  system.  These  briefings  invite  comment 
and  discussion.  After  the  briefings  any  necessary 
modifications  are  made  before  the  system  is  actually 
approved . 

Stage  four  involves  integrating  the  system  with  any 
existing  management  control  systems,  and  actual 
implementation  and  use. 

Stage  five  is  continuous  monitoring  of  the  final 
system.  This  provides  feedback  for  performance  improvement 
as  well  as  system  modification.  In  addition,  this  lends 
credibility  to  the  performance  improvement  program. 
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Then#  are  drawbacks  to  this  method.  Sink  said, 
'However,  difficulties  in  operationalizing  measurement 
systems  that  have  origins  in  a  participative  process 
hindered  early  efforts.  The  question  of  how  to  evaluate 
performance  against  a  list  of  measures  that  is  highly 
heterogeneous  became  critical  to  continued  development.  The 
MCP/PMT  overcomes  these  difficulties'  (28:276). 

MCP/PMT  is  similar  to  NP/PMM,  except  that  it  adds  a 
more  structured  approach  to  implementing  the  measures.  This 
approach  allows  the  aggregating  of  multiple,  heterogeneous 
measures  into  a  common  output. 

There  are  other  very  structured  methods  for  developing 
productivity  measures,  most  of  which  fit  into  one  of  Sink's 
broad  categories.  The  methodology  for  generating 
efficiency  and  effectiveness  measures  (MGEEM) ,  developed  by 
the  Air  Force  Human  Resources  Laboratory,  is  one  of  these 
systems  and  could  be  categorized  as  an  MCP/PMT  system. 

MGEEM  is  detailed  enough  to  be  used  by  nearly  anyone  with  a 
good  understanding  of  the  organization  to  be  measured. 

MGEEM  starts  by  forming  a  group  from  the  organization’s 
upper  management,  who  then  use  a  group  process  to  define 
the  measurable  facets  of  the  mission  of  the  organization 
under  study.  These  goals,  called  key  result  areas  (KRA’s) 
are  then  passed  to  another  group  from  middle  management. 
This  group  then  develops  specific  indicators,  or  areas  to 
be  measured.  These  measures  are  then  implemented  and 
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feedback  started.  MGEEM  also  Includes  Information  on 
problem  solving,  risks,  pitfalls,  fears  and  some  examples 
(33)  . 

Another  valuable  tool  for  helping  to  develop  a 
measurement  system  is  to  look  at  the  organization  with  a 
systems  approach.  This  can  help  develop  measures  by 
clarifying  the  inputs  and  outputs  ( 18 : 32 ; 34 : 2 13)  .  All  of 
these  approaches  involve  the  worker  in  choosing  the  items 
to  be  measured.  Involving  the  worker  is  important  to 
assuring  organizational  validity. 

Measurement  System  Design  Basics 

Getting  the  employees  involved  in  designing  the 
measurement  system  may  be  the  most  important  step  in 
implementing  a  performance  improvement  system.  If  the 
employees  are  not  convinced  that  the  measurement  system  is 
valid,  then  no  effort  will  really  be  successful  in 
implementing  that  system.  In  addition  to  fostering 
organizational  validity,  getting  the  employees  involved 
also  causes  them  to  focus  on  performance.  This  often  causes 
them  to  improve  their  own  performance  and  bring  to 
attention  hindrances  to  performance  that  exist  in  the 
system. 

It  appears,  from  the  number  of  times  mentioned  in  the 
literature  by  authors  such  as  Sink,  Lemonias  and  Usilaner, 
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and,  Bolta,  that  the  three  most  Important  parts  in  the 
development  of  a  performance  measurement  system  are  top 
management  support,  user  acceptance,  and  simplicity  of  use. 
Top  management  support  is  essential  or  the  people  will 
simply  not  see  the  need  to  be  involved  in  the  program.  This 
management  support  helps  foster  an  organizational  climate 
of  performance  improvement.  Furthermore,  if  top 
management  is  not  committed  to  productivity  improvement  and 
measurement,  any  time  spent  on  system  development  will 
probably  be  wasted. 

User  acceptance  can  be  fostered  in  many  ways.  Without 
user  acceptance  the  system  will  never  work  well,  because 
the  system  will  inevitably  require  inputs  from  the 
employees.  If  the  employees  do  not  accept  the  system,  those 
inputs  may  not  be  provided  or  may  be  inaccurate.  In 
addition  to  simply  accepting  the  measurement  system,  the 
users  need  to  accept  the  idea  of  performance  improvement. 

Simplicity  is  a  relative  thing.  The  users  need  to  be 
able  to  understand  what  is  being  measured  and  why.  In 
addition,  they  need  to  understand  the  results,  and  how  to 
affect  changes  in  those  results. 

After  a  good  performance  measurement  system  is 
designed,  the  organization  can  start  focusing  on  what  can 
be  done  to  improve  performance.  During  the  improvement 
stage,  workers  and  managers  are  often  given  added  training. 
This  training  may  be  a  refresher  in,  or  continuation  of 


that  training  the  organization  normally  gives,  or  it  could 
be  new  techniques  in  management,  or  problem  solving  from 
some  source  outside  of  the  organization. 

Often,  a  management  action  team  is  formed  to  solve 
problems.  If  this  is  the  case,  these  people  are  given 
special  problem  solving  training,  and  training  in 
analytical  techniques  (3:136). 

The  last  step  is  follow-through.  A  performance 
improvement  system  needs  constant  monitoring  in  order  to 
provide  management  the  inputs  necessary  to  keep  performance 
improving.  If  the  system  is  not  maintained  and  used,  it  is 
useless . 

Public  Sector  Productivity 

The  general  public  opinion  of  the  government  is  that 
it  is  not  as  efficient  as  it  should  be  (9:253).  Almost 
everyone  agrees  that  part  of  the  reason  is  that  the 
government  has  too  much  red  tape.  In  many  ways,  it  appears 
that  government  in  general,  and  the  federal  government 
specifically,  needs  performance  improvement  programs  to 
keep  from  stagnating  and  becoming  too  encumbered  with 
excess  rules  and  regulations. 

In  the  private  sector  the  inefficient  companies  go  out 
of  business.  If  a  company  becomes  unproductive,  it 
probably  will  be  warned  when  the  profits  go  down.  If  that 
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company  fails  to  react  in  a  timely  and  appropriate  manner, 
then  its  early  demise  is  nearly  assured.  Government  has  no 
simple  items  like  profit,  or  products  produced  per  man¬ 
hour,  to  watch.  In  the  private  sector  companies  are  in 
direct  competition  with  each  other.  This  competition  is 
often  lacking  in  the  federal  government  (22:14).  Instead, 
it  would  appear  that  other  methods  must  be  developed  to 
provide  the  same  type  of  information.  One  could  gauge  how 
the  federal  government  is  doing  by  how  much  one  gets 
compared  to  how  much  tax  one  pays.  But  that  would  give  too 
many  opinions  t ..fro  are  dependent  on  too  many  factors.  So 
the  government,,  unlike  private  industry,  finds  itself 
without  a  convenient  method  of  determining  organizational 
performance.  This  can  be  remedied  with  a  performance 
improvement  system. 

Often  an  organization  could  reap  great  benefits  from 
just  designing  a  performance  measurement  system,  even  if  it 
were  never  implemented.  Because,  when  a  performance 
measurement  system  is  designed,  it  causes  people  to  look  at 
priorities  and  management  basics.  This  design  process 
causes  management  to  identify  unnecessary  work.  When  the 
system  is  implemented  and  people  start  looking  past  the 
obvious  for  ways  to  improve,  then  less  obvious 
inefficiencies  start  to  be  identified.  This  process  is 
necessary  in  any  organization,  but  even  more  so  in  an 
organization  as  prone  to  picking  up  excess  rules  and 
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procedures  as  the  federal  government.  Once  the  system  of 
productivity  measurement  is  in  place,  and  the  major 
problems  have  been  worked  out,  people  can  continue  to 
search  for  areas  to  improve.  The  engineering  department  of 
the  city  of  Calgary,  Canada,  found  that  this  search  for 
improvement  can  go  all  the  way  to  the  'very  bedrock  of  the 
organizational  structure*  (27:144). 

Once  all  the  problems  have  been  worked  out,  it  is 
expected  that  competition  with  other  organizations,  or  pa3t 
history,  will  keep  the  inefficiencies  from  building  back 
up.  However,  this  has  never  been  shown,  because  no  one  has 
ever  really  gotten  all  the  way  to  the  end.  Nor  does  this 
author  believe  the  end  will  ever  be  reached,  because  this 
is  a  process  that  needs  to  continue.  Once  the  performance 
improvement  system  is  in  place  and  has  the  support  it 
needs,  then  it  becomes  a  perpetual  self  cleaning  process 
(9:256).  Removing  unnecessary  hindrances  and  regulations 
is  a  process  that  is  vital  and  necessary  for  any 
organization  to  survive,  and  thrive,  especially  government. 

USAF  Productivity  Programs 

The  United  States  Air  Force  has  at  least  two  major 
programs  on  productivity  improvement.  One  that  has  already 
been  mentioned  is  MGEEM,  and  it  is  available  as  a  resource 
upon  request  (33).  Another,  used  by  the  Tactical  Air 
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Command  (TAC)  is  the  PEER  competition  (30) . 

M3EEM  specializes  in  unique  individual  development  of 
performance  measurement  systems.  It  includes  a  series  of 
pamphlets  that  aid  in  the  development  of  such  systems  using 
a  modified  MCP/PKfT  as  described  earlier.  This  has  been 
especially  developed  to  take  care  of  some  of  the  unique 
problems  encountered  in  the  USAF  (33) . 

The  PEER  competition  used  by  TAC  is  something  that  an 
outside  contractor  was  hired  to  develop.  The  contractor 
produced  a  few  performance  measures  for  each  organization 
on  base.  Each  base  in  TAC  is  rated  using  these  measures. 
Between  1978  and  1984  TAC  improved  the  sortie  rate  and  in 
service  aircraft  rate  by  80  percent.  General  Creech,  then 
Commander  of  TAC,  attributed  that  improvement  to 
eliminating  performance  barriers  by  decentralization,  and 
to  competition  between  units  (22:14).  Brigadier  General  Roy 
M.  Goodwin,  Deputy  Chief  of  Staff  for  Engineering  and 
Services  for  TAC,  said  that  two-thirds  of  the  indicators 
presently  being  measured  are  increasing.  The  bases  that 
have  consistently  poor  performance  on  any  one  of  the 
indicators  get  looked  at  carefully,  and  usually  have  some 
problem  that  gets  solved  because  of  this  attention.  In  this 
way  the  PEERS  competition  allows  the  command  structure  to 
identify  problem  areas,  and  also  to  highlight  strengths  (11). 

Baumgartel  and  Johnson  attempted  to  measvre  the 
performance  of  base-level  AFCE ,  based  on  goal  attainment.  A 
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model  was  developed  using  inputs  and  outputs  available  in 
the  base  engineering  automated  management  system  (BEAMS) 
and  the  command  update.  However,  they  found  that  more 
research  would  need  to  be  done  to  get  output  measures  to 
reflect  goal  attainment,  because ,  although  specific  input 
data  was  available,  the  same  was  not  true  of  the  output 
data  (2) .  The  output  data  could  not  be  tied  directly  to  one 
specific  goal.  This  model  identified  seven  main  goals  for 
the  engineering  and  environmental  planning  branch,  with  18 
sub-goals.  They  noted  that,  in  most  cases,  the  goals  were 
not  specific  enough  to  allow  direct  measurement.  The  goals 
identified  for  the  Engineering  and  Environmental  Planning 
Branch  include  the  following  (2:81-83) : 


Goal  »1--Faclllty  Life  Cycle  Cost 

A.  Identify  and  program  MCP  projects,  and  monitor 
approval,  design  and  construction  phases  to  ensure 
maximum  durability  and  maintainability  of  accepted 
facilities . 

B.  Ensure  in-house  design  complies  with  AFM  88-15  and 
applicable  building  codes. 

Goal  »2--Faoility  Function 

A.  Ensure  new  construction  projects  are  identified  and 
programmed  in  a  timely  manner,  and  are  designed  and 
located  in  accordance  with  the  user’s  requirements. 

B.  Identify  and  program  contract  corrections  to 
facilities  which  are  functionally  inadequate 
for  mission  requirements. 
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Goal  »3--Faci 1 i tv  Protection 

A.  Ensure  corrective  contract  actions  for  identified 
facility  fire,  safety,  and  security  deficiencies  are 
programmed,  designed,  and  completed  in  a  timely 
manner . 

B.  Ensure  new  contract  work  complies  with  regional 
requirements  for  structural  protection  against 
weather  and  earthquake-related  forces. 

Goal  »4--Utility  and  Energy  Supply  and  Conservation 

A.  Complete  engineering  analyses  of  existing  and 
programmed  utility  supply  and  distribution  systems 
to  identify  inadequate  supply  or  inefficient 
operations . 

B.  Ensure  new  facilities  are  designed  and  constructed  to 
minimize  energy  consumption. 

C.  Complete  engineering  analyses  of  existing  facilities 
to  identify  sources  of  energy  waste,  and  program 
projects  to  correct  deficiencies  identified. 


Goal  »5- -Environmental  Protection  and  Conservation 

A.  Ensure  facility  projects  are  assessed  for  adverse 
environmental  impact  prior  to  programming. 

B.  Include  environmental  impact  considerations  during 
master  planning  actions,  to  minimize  adverse  impact 
due  to  siting. 

C.  Ensure  control,  handling  and  disposal  of  hazardous 
substances  and  waste  products  comply  with  EPA 
standards . 

D.  Ensure  that  construction  practices  comply  with  EPA 
standards . 

Goal  »6- -Faci 1 i ty  Occupant /User  Requirements 

A.  Complete  architectural  studies  of  facilities  to 
identify  inadequate  aesthetic  conditions  and  facility 
deficiencies  contributing  to  occupant  discomfort. 

B.  Ensure  designed  projects  comply  with  applicable  life 
safety  and  public  health  code  requirements. 
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C.  Ensure  identified  facility  life  safety  and  health 
code  deficiencies  requiring  contract  corrective 
action  are  programmed,  designed,  and  completed  in  a 
timely  manner. 

D.  Identify,  program  and  specify  custodial  contracts 
required  for  base  facilities  and  ensure  contractor 
compliance  with  the  contractual  requirements. 


Goal  *7--0ther  Non-facility  Requirements 

A.  Provide  professional  architectural  and  engineering 
assistance  to  operations  branch  and  to  other 
organizations  as  required. 


Kaneda  and  Wallet  extended  the  research  done  by 
Baumgartel  and  Johnson,  and  developed  ratios  to  be 
measured.  These  ratios  were  tested  by  survey  to  determine 
their  usefulness.  Six  of  the  ratios  were  determined  to  be 
useful  in  measuring  performance  in  the  design  section.  In 
addition,  the  survey  results  revealed  that  although  AFCE 
managers  saw  the  need  to  measure  performance,  they  were 
opposed  to  the  idea  of  an  imposed  system.  AFCE  managers 
made  it  clear  from  the  results  that  they  wanted  any 
measurement  to  stay  at  the  organizational  level.  The  system 
needed  to  be  flexible  so  that  it  could  be  tailored  to 
individual  needs.  Kaneda  and  Wallett  also  warned  that 
comparing  design  sections  would  not  be  advisable  and  that 
the  productivity  measures  ’ . . .  must  be  a  means  to  improve 
management  rather  than  ends  in  themselves'  (17:78).  The 
ratios  developed  were  (17:63): 
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Total  estimated  dollar  amount  of  contract  projects  and 
in-house  work  orders  designed  divided  by  total  design 
man-hours . 

Total  number  of  projects  designed  (complete  and  ready  for 
acquisition  action)  divided  by  total  design  man-hours. 

Total  number  of  facility  inspections  and  utility  system 
surveys  completed  divided  by  total  man-hours  to  complete 
surveys  and  inspections. 

Total  estimated  dollar  amount  of  AE  design  acquisition 
packages  prepared  divided  by  the  total  man-hours  to 
prepare . 

Total  estimated  dollar  amount  of  contract  projects  and  in- 
house  work  orders  designed  divided  by  total  design  labor 
cost . 

Total  number  of  projects  designed  (complete  and  ready  for 
acquisition  action)  divided  by  total  design  labor  cost. 

Astin  and  Ruff  developed  a  model  to  compare  one  design 
section  with  another  using  constrained  facet  analysis  (CFA) 
(1).  'Constrained  Facet  Analysis  is  a  linear  programming 
model  designed  to  evaluate  the  relative  productivity--  or 
efficiency--  of  a  number  of  organizations  which  are  all 
producing  different  combinations  of  outputs  using  different 
combinations  of  the  same  inputs.  It  is  theoretically 
attractive  because  it  accomplishes  the  determination  of  the 
relative  efficiency  of  organizations  with  no  need  for  a 
priori  evaluations  of  the  relative  weights  or  values  of  the 
various  resources  consumed  and  outputs  produced.  The  model 
determines  these  efficiencies  based  solely  on  the  actual 
data  representing  the  input  and  output  quantities'  (24:1). 
This  model  used  six  input  measures  and  20  output  measures 


based  on  some  of  the  work  of  Kaneda  and  Wallet.  This  model 
only  tested  CFA’s  ability  to  distinguish  the  high 
performance  sections  from  others  with  lesser  performance. 
As tin  and  Ruff’s  model  successfully  showed  that  CFA  could 
distinguish  performance  differences  as  expected.  The 
measures  developed  are  as  follows  (1:39-41): 

INPUTS 

1 .  Labor  man-hours . 

2.  Labor  costs. 

3.  Years  experience. 

4.  Personnel  skill  level  aggregate. 

5.  Number  of  professional  education  courses  completed. 

6.  One  over  the  number  of  additional  duties  performed. 

OUTPUTS 

1.  Total  contract  funds  obligated. 

2.  Estimated  dollar  amount  of  all  projects  designed. 

3.  Total  O&M  maintenance  and  repair  project  funds 
obligated. 

4.  Total  O&M  minor  construction  funds  obligated. 

5.  Total  number  of  projects  designed. 

6.  Total  number  of  facility  inspections  and  surveys 

completed . 

7.  Total  number  of  special  technical  studies  and  reports 
completed . 

8.  One  over  the  total  funds  expensed  on  contract  change 
orders . 

9.  One  over  the  number  of  contract  change  orders. 

10.  Total  estimated  dollar  amount  of  in-house  work  orders 
designed . 

11.  Total  estimated  dollar  amount  of  architect-engineer 
packages  prepared. 

12.  Total  A-E  funds  obligated. 

13.  Estimated  dollar  amount  of  MCP  project  books. 

14.  Number  of  work  orders  reviewed  and/or  evaluated. 

15.  Number  of  technical  reviews  accomplished  on  designed 
projects . 

16.  Pages  of  project  specifications. 

17.  Total  number  of  oral  presentations  made. 

18.  Number  of  facility  surveys  completed. 

19.  Total  hours  of  surveys  completed. 

20.  Number  of  pages  of  engineering  drawings  completed. 
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In  a  dissertation  in  1986,  Rumsey  developed  and  tested 


the  technical  validity  of  a  similar  model  using  CFA. 
However,  he  found  that  the  results  were  less  than  favorable 
(24).  The  measures  Rumsey  used  were  the  following  (24:69): 
INPUTS 

1.  Military  engineers  assigned. 

2.  Civilian  engineers  assigned. 

3.  Other  military  assigned. 

4.  Other  civilians  assigned. 

5.  Total  years  experience. 

6.  Personnel  skill  level. 

OUTPUTS 

1.  Total  contract  funds  obligated. 

2.  Estimated  value  of  project  designs  completed. 

3.  Total  O&M  maintenance  and  repair  project  funds 
obligated . 

4.  Total  number  of  design  projects  completed. 

5.  Total  cost  of  contract  change  orders. 

6.  Total  number  of  contract  change  orders  designed 
during  this  period. 

7.  Total  estimated  dollar  amount  of  in-house  work  orders 
designed  during  this  period. 

8.  Estimated  value  of  architect-engineer 
packages  completed  during  this  period. 

9.  Estimated  value  of  MCP  project  books  completed  during 
this  period. 

10.  Number  of  work  orders  reviewed. 

11.  Number  of  technical  reviews  completed  on  design 
projects . 


White  Collar  Productivity 

Bumbarger  takes  exception  to  the  use  of  the  term  white 
collar;  he  prefers  to  look  at  the  amount  of  knowledge  work 
done  by  the  worker  (5:5).  For  this  work,  white  collar 
workers  will  be  simply  defined  as  those .who  do  almost 
entirely  knowledge  work. 
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Some  claim  white  collar  productivity  cannot  be 
measured  (4:47).  Bolte  aaid  that  this  is  not  true.  'All 
administrative  areas  produce  a  specific  end  product  or 
service  that  can  be  measured*  (4:47).  The  question  is  one 
of  accuracy,  and  willingness  to  expend  the  necessary  effort 
to  develop  a  method  of  measure. 

It  is  true,  however,  that  the  measurement  of  white 
collar  productivity  is  not  nearly  as  straight  forward  as 
blue  collar  productivity.  One  of  the  problems  for  white 
collar  areas  is  that  'statistical  techniques  commonly  used 
by  industrial  engineering  or  in  quality  circles  require 
quantitative  data,  which  is  not  easily  developed  in 
professional  or  admlnistrati ve  positions'  (13:288).  Klnlaw 
concluded  'it  is  also  apparent  that  development  of  models 
and  technical  descriptions  of  measures  for  complex  white 
collar  organizations  (e.g.,  scientific  and  engineering)  is 
still  in  the  formative  stage*  (18:30). 

One  of  the  problems  is  that  white  collar  workers 
generally  have  more  than  one  output,  and  the  outputs  often 
compete  directly  for  organizational  resource.  General ly , 
the  problem  is  that  the  inputs  and  outputs  are  not  as 
readily  recognized.  Yet,  with  some  work,  that  too  can  be 
overcome  and  is  addressed  specifically  by  many  of  the 
measurement  system  development  techniques  like  MGEEM. 
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One  of  the  ways  of  defining  input  and  outputs  is  to 
look  at  the  organization  with  a  systems  perspective.  By 


defining  boundaries  around  the  system,  recognition  of 
inputs  and  outputs  becomes  easier.  Inputs  and  outputs  have 
to  actually  cross  the  boundaries. 

Tuttle  and  Romanowski  divided  white  collar  workers 
into  two  groups,  direct  outcome  and  indirect  outcome.  "In  a 
direct  outcome  system,  there  is  a  direct,  high  probability 
relationship  between  the  output  and  the  outcome*  (34:214). 
Clerks  might  fit  in  the  direct  outcome  category;  however, 
engineers  fit  in  the  indirect  outcome  category.  This  means 
that  even  though  an  engineer  is  efficiently  using  his 
resources,  and  is  being  productive  with  his  time,  he  could 
still  fail  to  meet  organizational  goals,  and  therefore  be 
ineffective . 

Engineering  Productivity 

In  a  study  of  productivity  measurement  for  research 
and  development  engineers,  Schainblatt  concluded,  'There 
are  no  currently  used  systems  for  measuring  the 
productivity  of  scientific  and  engineering  groups  without 
substantial  flaws.  Nor  does  the  literature  on  productivity 
measurement  offer  encouragement  that  suitable  systems  will 
soon  be  available’  (25:10).  If  measuring  the  productivity 
of  research  and  development  (R&D)  functions  is  an 
underdeveloped  field,  design  engineering  may  be  even  more 


so.  The  literature  contains  great  quantities  of  information 
on  productivity,  large  amounts  of  information  on  white 
collar  productivity  and  some  information  about 
professionals.  However,  outside  of  the  few  articles  about 
H&D ,  there  is  only  occasional  mention  of  engineering. 
Unfortunately,  the  information  about  R8tD  is  not  very 
applicable  to  AFCE. 

By  definition  most  engineers  are  white  collar  workers. 
Thus,  most  of  what  was  said  about  white  collar  workers  may 
apply  here  also.  There  are  some  peculiarities  about 
engineering  that  make  things  different.  The  first 
difference  is  a  very  strong  resistance  among  engineers  to 
performance  measurement  (18:32).  This  stems  from  the 
nature  of  the  work.  Another  difference  is  that  the  most 
important  measurement  for  engineering  is  effectiveness  (3). 
That  is  to  say,  meeting  organizational  goals. 

Unfortunately,  effectiveness  can  be  hard  to  measure.  This 
is  especially  true  in  the  design  section,  where  one  of  the 
goals  is  to  have  a  design  work  well  when  construction  is 
finished;  yet,  there  may  be  many  years  between  design  and 
completion  of  the  construction.  In  light  of  the  difficulty 
in  measuring  effectiveness,  many  managers  have  settled  for 
something  else--often,  something  like  meeting  a  schedule. 
This  can  conflict  with  the  engineers’  sense  of 
professional  ethics,  if  they  sees  this  as  forcing  low 
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quality  or  possibly  unsafe  work  to  be  released  just  to  meet 


a  schedule. 

If  the  performance  of  engineers  is  truly  to  be 
measured,  some  method  of  measuring  effectiveness  needs  to 
be  developed  first,  and  maintained  as  the  primary  measure. 
'In  this  type  of  system  (indirect  outcome),  the  most 
relevant  factors  are,  at  least  initially,  effectiveness  and 
quality.  Only  after  it  has  been  demonstrated  that  the 
correct  outputs,  of  appropriate  quality,  are  being  produced 
should  efforts  be  focused  on  improvements  in  productivity 
and  efficiency  (34:215).' 

Another  important  area  for  engineering  is  simply 
eliminating  unnecessary  work.  Often  a  company  can  get 
clerks  or  technicians  to  do  the  work,  or  simply  remove  the 
work  altogether. 

Summary 

Productivity  improvement  is  important  to  the  nation. 
The  federal  government  is  no  exception  to  this  need. 
Productivity  improvement  systems  require  several  things. 
These  requirements  are  a  management  focal  point,  top  level 
support  and  commitment,  a  written  plan  with  objectives  and 
goals,  a  meaningful  performance  measurement  system,  use  of 
the  plan  to  hold  managers  accountable,  awareness  of  the 
importance  of  productivity,  and  ongoing  problem 
identification  and  solution.  A  lack  in  any  of  these  areas 


will  lessen  the  capabilities  of  the  performance  improvement 
system. 

The  three  most  important  parts  of  a  productivity 
improvement  program  are  management  commitment,  a  good 
measurement  system,  and  follow-through.  Without  management 
commitment  to  change,  there  will  be  no  system.  Follow- 
through  is  necessary  to  retain  any  improvements,  and  a  good 
measurement  system  is  vitally  important. 

The  measurement  system  must  be  accurate  in  its 
assessment  of  productivity  and  needs  to  be  trusted  by  the 
employees.  If  the  employees  don’t  trust  the  system,  it 
will  probably  not  succeed.  However,  the  performance 
measurement  system  need  not  be  perfect. 

There  are  no  easy  victories,  or  instant  successes  in 
this  business.  The  improvements  in  productivity  come  from 
the  application  of  good  sound  management  practices.  The 
difference  between  performance  improvement  systems  and 
other  management  methods  is  that  with  a  measurement  system 
and  a  focus  on  performance,  people  can  intelligently  work 
toward  the  improvement  of  organizational  performance. 

Performance  improvements  come  from  several  sources. 
First,  as  people  focus  on  the  mission  of  their 
organization,  they  eliminate  or  place  less  emphasis  on 
certain  tasks.  Improved  management  practices  also  boost 


41 


performance.  After  all  of  that,  people  may  simply  work 
harder  because  of  the  interest  given  to  performance. 

The  performance  of  white  collar  workers  is  hard  to 
measure,  and  an  engineer’s  performance  is  even  harder  to 
measure.  The  things  that  are  the  most  important  in  order 
to  successfully  measure  performance  of  engineers  are 
effectiveness,  quantity,  quality,  and  value. 

Performance  improvement  is  a  lot  of  work  and  comes 
with  risks.  To  effectively  improve  performance,  management 
must  be  committed  to  change.  They  must  also  be  willing  to 
commit  the  effort  needed  to  get  and  maintain  that  change. 
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III.  Case  Study 


Management  Environment 

Tinker  AFB  is  an  Air  Logistics  Center  (ALC)  and  part  of 
the  Air  Force  Logistics  Command  (AFLC) .  As  such  it  is  by 
nature  a  very  large  base,  with  a  total  population  in  excess 
of  25,000  employees  (35:c-30).  The  engineering  design 
section  has  been  broken  into  two  parts.  One  serves  only  the 
ALC  maintenance  function;  the  other  serves  the  normal 
function  of  a  design  section,  serving  the  rest  of  the  base. 
This  study  pertains  only  to  the  design  section  serving  the 
main  base,  the  more  typical  design  function. 

The  design  section  has  at  least  three  main  functions: 
design,  staff  support  for  management,  and  engineering 
support  for  AFCE  maintenance  functions.  Design  is  the 
primary  reason  for  the  existence  of  the  design  section. 
Design  can  be  accomplished  one  of  three  ways:  in-house, 
Architect-Engineer  (AE)  contract,  or  the  Army  Corps  of 
Engineers  (COE).  AE  designs  still  require  in-house  support 
and  inspection.  AE  contracts  take  approximately  40  percent 
of  the  time  that  would  be  used  for  an  in-house  design  (14) . 
This  time  is  spent  mostly  on  inspection  of  the  design.  COE 
designs  take  much  less  time  from  the  in-house  force,  but 
still  require  the  completion  of  design  guidance,  and 
several  design  reviews. 
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Staff  support  includes  analysis  of  suggestions  for  the 
Air  Force  suggestion  program,  occasional  reports,  and  other 
engineering  analysis.  The  support  for  the  AFCE  maintenance 
function  requires  more  time.  This  support  includes 
analysis  of  various  system  problems,  such  as  air 
conditioning  or  electrical  distribution  problems.  All  of 
the  utility  systems  on  base  require  some  engineering 
support . 

The  design  section  at  Tinker  AFB  is  staffed  with 
approximately  30  engineers  and  architects  to  handle  these 
tasks.  This  staff  is  divided  into  groups  by  specialty, 
mechanical,  electrical,  civil  and  architectual  engineering. 
These  groups  each  have  an  experienced  engineer  as  the  group 
leader.  Each  group  leader  is  responsible  for  the 
coordination  and  completion  of  the  work  his  group 
accomplishes . 

Tinker  AFB  was  also  a  test  site  for  the  WIMs  system. 
This  system  was  installed  about  1982  and  was  specifically 
designed  for  AFCE.  In  addition  the  design  section  has  just 
recently  received  a  computer-aided  drafting  (CAD)  system. 
This  system  was  purchased  by  AFLC  for  each  of  the  bases  in 
that  command.  The  training  for  the  CAD  system  was  still 
being  provided.  Management  had  decided  to  use  the  training 
provided  in  a  rather  unusual  way,  by  training  the  engineers 
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first  and  allowing  them  to  train  the  draftsmen.  This 
approach  seemed  to  be  working  quite  well  (16). 

In  addition  to  the  command-purchased  equipment,  the 
design  section  tried  to  make  adequate  use  of  any  classes 
available  to  them.  Specifically,  they  made  use  of  the 
classes  at  the  Air  Force  Institute  of  Technology’s  School 
of  Civil  Engineering  and  Services. 

Research  Findings 

The  system  history  is  very  simple.  When  the  design 
chief  took  charge  six  years  ago,  no  formal  system  was 
passed  on.  That  is  not  necessarily  to  say  that  the 
previous  design  chief  did  not  measure  productivity,  but 
that  whatever  measurement  system  was  used  left  with  him. 

Most  of  the  measures  were  implemented  upon  the  arrival 
of  the  new  design  chief.  There  was  no  formal  technique  used 
to  develop  this  system;  little  discussion,  outside  of 
required  responses,  seems  to  have  taken  place. 

The  system  was  mostly  developed  by  the  design  chief. 
There  were  not  any  group  techniques  involved.  The  system 
did  not  need  any  group  consensus,  because  the  measures  were 
only  intended  to  keep  the  design  chief  informed.  The  system 
has  been  quite  evolutionary  in  nature,  with  some  measures 
being  added  over  time,  while  others  have  been  tried  and 
later  dropped. 
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As  in  the  development,  no  special  effort  was  made  to 
aid  in  implementation.  Implementation  was  simply  directed 
by  the  design  chief.  The  only  resistance  he  had  in  any 
implementation  was  when  he  decided  to  use  suspense  slips  to 
monitor  all  the  responses  required  from  his  section.  Each 
piece  of  correspondence  that  requires  any  response  at  all 
is  recorded  in  this  system.  That  way  the  design  chief  is 
made  aware  of  anything  that  does  not  get  done  on  time. 

System  Presentation 

The  section  under  investigation  was  chosen  because  of 
a  computer  program  they  use  that  has  some  performance 
measures  built  into  it.  This  program’s  real  function  is  to 
produce  a  design  schedule.  This  computer  program  forms  the 
backbone  of  the  performance  measurement  and  control  system, 
but  there  are  other  measures  used.  However,  there  is  not  a 
uniform  use  or  understanding  of  those  measures,  or  of 
performance  in  general.  The  method  of  performance 
measurement  used  is  not  really  a  system.  It  is  more  of  a 
collection  of  independent  measures  that  function,  and  are 
used  independently. 

The  section  under  study  measures  effectiveness, 
quality,  productivity,  and  efficiency.  Each  of  these  areas 
will  be  described  here.  Functional  definitions  as  well  as 
methods  of  measurement  will  be  given.  A  brief  overview  is 
shown  below. 
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Ef  f  ectiyeness 


Completion  of  design  schedule 
Negative  feedback 

Backlog  of  work  orders  and  cost  estimates 
Ef  f iciency 

Design  schedule  program--Perf ormance  against  schedule 

Productivity 
Time  sheets 

Percentage  of  time  spent  on  design 

Quality/ Value 
Inspection  of  work 

Ef f ectiveness .  Effectiveness  was  pointed  out  by  the 
literature  review  as  the  single  most  important  measure  for 
engineers.  Effectiveness  is  a  very  difficult  thing  to 
functionally  define.  The  standard  definition  is  meeting 
organizational  goals.  The  goal  of  the  design  chief  was  to 
satisfy  the  chain  of  command.  There  are  two  main  priorities 
for  the  design  section  (14).  The  first  priority  is  to 
design  projects.  The  required  quantity  of  projects  is 
variable  and  set  by  the  major  command.  This  is  done  by 
somehow  combining  the  information  in  the  Civil  Engineering 
Contract  Reporting  System  (CECORS)  with  the  command  funding 
ability  for  the  year.  This  gives  an  annual  target  of  total 


estimated  contract  cost.  If  that  amount  of  design  is  done, 
the  organization  met  that  goal.  That  goal  can  be  exceeded 
if  additional  designs  are  completed  to  take  advantage  of 
any  extra  money  that  is  made  available  at  the  end  of  the 
year . 

The  second  priority  is  to  "stay  out  of  trouble.’  This 
is  not  unusual  in  a  military  organization,  and  probably 
normal  in  many  other  organizations.  This  goal  is  to  do 
everything  else  required,  well  enough  that  the  commander 
does  not  have  problems  brought  to  his  attention  (14). 

These  things  may  be  trivial  or  important,  but  they  are  all 
secondary  to  design.  Although  the  second  goal  is  hardly 
what  the  literature  would  recommend,  it  is  functional  and 
agreed  upon  by  the  squadron’s  management  (14; 32). 

The  squadron  commander  measured  the  effectiveness  of 
the  design  section  by  its  ability  to  meet  command  design 
targets  and  take  advantage  of  year-end  money.  Secondly,  he 
saw  them  as  effective  because  there  were  not  any  negative 
reports  about  the  design  section.  He  did  not  formally 
track  any  of  this  information  (32) . 

The  squadron  commander  was  quite  satisfied  with  the 
performance  of  the  design  section.  One  other  factor  that 
particularly  pleased  him  was  the  ability  of  the  design 
section  to  take  in  stride  changes  to  the  design  schedule 
(32).  This  could  be  attributed  to  three  things:  a  flexible 
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attitude,  good  management,  and  the  computer  program  to 
develop  design  schedules. 

The  method  of  measuring  effectiveness  is  not  really 
what  was  expected.  However,  it  does  work  for  the  design 
section  chief,  and  it  provides  the  small  amount  of 
refinement  he  needs  to  keep  control. 

The  one  major  item  used  to  assure  effectiveness  is  the 
design  schedule.  This  is  monitored  on  a  weekly  basis.  The 
design  schedule  is  watched  closely  by  the  chief  of  design 
and  all  of  the  group  leaders,  and  is  the  only  item  reported 
outside  of  the  section. 

The  other  part  of  the  effectiveness  measurement  system 
is  to  track  work  coming  and  going.  All  items  that  come  into 
the  section,  that  require  any  response,  are  given  a 
suspense  slip.  Those  suspenses  are  tracked  to  assure  that 
the  work  is  eventually  completed.  In  addition,  work  orders 
and  cost  analyses  that  require  design  input  are  tracked 
both  in  quantity  and  timeliness.  High  priority  work  orders 
or  cost  estimates  are  given  a  management  push;  otherwise, 
these  things  are  pushed  as  necessary  to  keep  from 
developing  too  much  of  a  backlog  (15). 

Ef f iciency .  Efficiency  is  measured  by  the 
computerized  design  schedule  program.  This  is  done  by 
simply  comparing  the  estimated  hours  for  design  with  the 
actual  hours  used  to  design  a  project  (21) .  This  is  done 
at  various  stages  of  design.  It  is  monitored  on 
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approximately  a  weekly  basis  by  the  chief  of  design  and  the 
group  leaders  (14).  There  is  one  admitted  weakness  of  this 
method  of  measure  and  that  is  estimating  the  hours  of 
design  time  (14;  20).  This  is  done  by  two  methods.  First, 
an  engineer  estimates  how  long  it  will  take,  then  that 
estimate  is  compared  to  the  six  percent  fee  that  an  A/E 
firm  would  receive.  The  AE  fee  is  calculated  by  taking  six 
percent  of  the  estimated  contract  price,  dividing  by  two  to 
separate  labor  and  overhead,  and  then  dividing  by  an 
average  of  >25  an  hour.  This  gives  a  rough  estimate  of  the 
number  of  hours  required  by  an  AE  to  do  the  same  project. 
These  two  figures  are  compared,  then  one  is  chosen  and 
loaded  in  the  design  schedule  (14).  Because  of  the  weakness 
in  the  estimate,  it  is  used  as  a  guideline  more  than  an 
absolute.  This  system  gives  averages  and  allows  the  section 
chief  to  keep  the  design  schedule  on  time.  The  aggregated 
figures  on  how  an  engineer  does  are  used  for  information  to 
back  up  his  performance  rating. 

Productivity .  Productivity  is  measured  with  three 
methods.  First,  a  time  sheet  is  filled  out,  on  a  daily 
basis  by  each  engineer.  The  time  sheet  explains  what  the 
engineer  did  with  his  time.  Some  amount  of  time  is 
expected  to  be  spent  on  non-design  items,  but  the 
percentages  spent  in  different  areas  are  monitored. 
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The  second  method  is  the  goal  of  having  40  percent  of 
the  time  spent  on  design.  This  is  tracked  individually,  by 
group  and  for  the  section  as  a  whole.  The  goal  is  not  to 
exceed  40  percent  but  to  stay  near  40  percent.  This  goal 
was  thought  to  be  in  the  Air  Force  regulation  somewhere, 
but  no  one  was  sure  where.  The  researcher  was  unable  to 
locate  it. 

Qual i ty .  The  chief  of  design  was  concerned  by  his 
inability  to  measure  quality.  He  was  considering  getting 
feedback  from  the  base  working  group  panel ,  which  is  a 
locally  run  organization  that  has  representatives  from  each 
of  the  organizations  on  base.  He  did  realize  that  the  kind 
of  feedback  he  would  receive  was  not  going  to  be  completely 
accurate  and  would  probably  be  negatively  skewed.  But  this 
weak  information  was  perceived  as  being  better  than  no 
information  at  all. 

The  interesting  observation  about  quality  is  that  the 
group  leaders  did  not  have  a  problem  defining  it,  nor  did 
they  have  a  problem  knowing  if  quality  was  there.  The 
group  leader’s  Job  includes  checking  each  of  the  projects 
an  engineer  does,  at  various  levels  of  completion.  When  the 
work  is  checked,  the  group  leader  assures  himself  that 
adequate  quality  is  there.  If  a  the  project  is  found  to  be 
lacking,  then  the  project  is  either  sent  back  or  notes  are 
made  as  to  what  needs  to  be  changed  for  the  next  review 
(16;  7).  While  it  is  true  that  there  is  no  written 
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standard  for  quality,  each  group  leader  had  firmly  in  mind 
what  quality  meant  to  him  and  checked  for  it.  This  measure 
was  not  quantified  in  any  way.  The  group  leaders  simply  did 
not  release  a  project  until  the  necessary  quality  was 
there.  The  researcher  believes  that  this  method  will  work 
fine  as  long  as  the  group  leaders  share  the  organizational 
views  of  quality  and  value.  Additionally  the  group  leaders 
must  be  capable  of  recognizing  quality  and  providing 
assistance  when  necessary,  to  assure  quality.  This  method 
is  arbitrary,  but  usable. 

An  additional  method  that  had  been  tried  and  abandoned 
was  tracking  change  orders.  This  practice  was  largely 
abandoned  after  a  short  time  because  there  were  so  few 
change  orders,  and  most  were  caused  by  other  than  quality 
issues.  This  alone  would  seem  to  indicate  that  adequate 
quality  was  included  in  the  designs. 

Research  Sequence 

Initially,  contact  was  made  with  a  design  section  with 
the  intention  of  both  implementation  and  study  of  a 
measurement  system.  Instead,  the  initial  contacts 
uncovered  a  system  in  the  process  of  being  implemented,  and 
a  second  section  (at  Tinker  AFB)  which  was  already  using 
the  same  program. 
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The  first  interviews  were  somewhat  hindered  by 
vocabulary  barriers.  The  chiefs  of  design  have  had  little 
formal  training  on  performance  measurement,  and  they  have 
little  understanding  of  the  in-depth  technical  issues. 

This  led  to  several  false  starts  due  to  different 
definitions  of  the  same  words. 

The  first  plan  for  study  was  to  include  both  of  the 
design  sections  previously  discussed.  It  soon  became 
apparent  that  the  section  currently  implementing  the 
measurement  system  would  not  be  ready  in  time  to  complete 
this  research.  That  forced  the  researcher  to  focus  on  the 
section  at  Tinker  AFB ,  which  had  been  using  the  program  for 
several  years. 

The  case  study  was  conducted  by  traveling  to  Tinker 
AFB  to  conduct  interviews  with  the  appropriate  people, 
using  a  semi -structured  interview  format.  Telephone 
follow-ups  were  also  conducted.  The  first  day,  the  chief 
of  design  was  interviewed  along  with  one  of  his  more 
involved  group  leaders.  Then  some  revisions  were  made  to 
the  interview  format.  The  second  day,  the  remaining 
available  group  leaders  were  interviewed.  The  last  day, 
the  squadron  commander  and  the  industrial  engineer  were 
interviewed . 
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Productivity  Measurement  System  Use 

The  system  was  developed  to  aid  the  chief  of  design  in 
monitoring  and  controlling  the  section,  and  that  is  all  it 
is  used  for.  The  system  is  only  used  by  the  chief  of 
design  and  the  group  leaders.  There  is  no  effort  at 
improvement  of  performance,  only  on  keeping  performance 
above  set  limits,  and  meeting  set  goals.  In  addition,  the 
system  is  not  used  directly  for  performance  appraisal. 
Performance  measures  of  each  engineer  are  taken  into 
account  for  appraisal,  but  not  used  directly. 

The  performance  measures  used  by  the  design  section 
are  not  thought  of  as  a  system,  although  they  are  used  in  a 
systematic  way  by  the  design  chief  to  control  the  section. 
The  system  represents  more  of  a  collection  of  independent 
measures,  each  used  in  a  slightly  different  way  to  control 
the  various  aspects  of  performance. 


Val idi ty 

The  investigation  of  validity  did,  as  mentioned  in  the 
methodology,  include  technical  validity,  organizational 
validity  and  use. 

Technical  validity  requires  the  system  to  accurately 
measure  the  areas  it  purports  to  measure.  The  system  does 
measure  the  right  things,  and  in  a  reasonably  correct 
manner.  However,  accuracy  is  questionable.  The  measures 


54 


of  effectiveness  and  quality  are  subjective,  and  the 
measure  of  efficiency  is  of  only  questionable  accuracy. 
However,  because  of  the  way  the  system  is  used,  the 
accuracy  required  is  also  low.  Therefore,  even  though  the 
system  is  not  highly  accurate,  it  does  meet  the  needs  the 
organization  imposed  upon  it.  The  system  seems  to  assure 
reasonable  levels  of  organizational  performance.  However, 
the  measurements  that  are  used  are  not  highly  refined. 

Technical  validity  of  the  organizational  model  is 
based  on  ease  of  measurement,  selection  of  appropriate 
variables,  reliability  and  validity  of  the  underlying 
variables.  The  validity  of  this  model  appears  to  be 
adequate.  The  variables  measured  reflect  the 
recommendations  of  the  literature  and  the  method  of 
measuring  each  variable  is  simple  and  straight  forward.  In 
addition  the  system  is  easy  to  use  and  very  well 
incorporated  in  the  management  structure.  However,  the 
efficiency  and  quality  measures  must  be  viewed  as  having 
only  questionable  validity. 

Organizational  validity  is  a  measure  of  how  well  the 
system  is  accepted  by  the  organization.  In  the  case  of 
this  system,  acceptance  by  the  user  is  rather  a  mute 
point.  The  system  is  used  mostly  for  control ,  not  for 
official  reports;  therefore,  no  real  threat  is  imposed. 
With  the  real  purpose  of  the  system  being  control,  the 
system  is  not  visible  to  anyone  other  than  management. 
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What  visibility  does  exist  is  accepted  simply  as  necessary 
management  control. 

The  system  has  been  effectively  used  to  do  what  the 
design  chief  wants.  This  is  not,  nor  was  it  intended  to 
be,  a  system  of  measures  designed  to  bring  systematic 
improvement  to  performance.  Rather,  the  system  was 
designed  to  keep  performance  in  bounds.  Specifically,  the 
system  is  in  existence  only  to  meet  the  management  need  to 
provide  control,  which  it  does  well  enough  to  satisfy 
management’s  expectations . 

Productivity  Improvement 

Some  of  the  biggest  possibilities  for  productivity 
improvement  have  not  been  used  here.  In  the  literature  the 
two  greatest  areas  for  productivity  improvement  come  from 
identifying  and  eliminating  unnecessary  work,  and  changing 
policies  and  procedures  that  hinder  productivity.  The 
section  under  study  has  not  visibly  done  either. 

As  far  as  changing  policies  and  procedures,  the 
section  seems  to  have  no  special  effort  to  do  this.  Part 
of  the  reason  for  this  is  undoubtedly  due  to  the  typical 
perception  that  it  would  take  a  lot  of  time  and  probably 
not  be  successful  anyway.  Instead,  the  section  here  has 
worked  within  the  confines  of  the  broader  organizational 
system. 
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When  asked  what  he  had  done  to  improve  performance, 
the  chief  of  design  mentioned  sending  himself  and  one  group 
leader  to  available  management  classes.  Unfortunately  the 
other  group  leaders  were  not  allowed  to  attend,  because  of 
a  rule  in  the  base  personnel  system  that  required  them  to 
be  designated  as  supervisors.  The  chief  of  design 
expressed  his  frustration  at  this  problem,  and  his 
inability  to  get  anything  done  about  it  (14). 

Summary 

The  model  of  performance  that  is  used  by  the  section 
under  study  does  provide  them  some  level  of  performance 
measurement.  The  problem  with  their  system  is  that  it  uses 
mostly  subjective  measures  and  little  actual  quantifiable 
data.  Although  this  system  could  be  improved,  there  is 
presently  little  desire  to  do  that.  The  system  in  use 
fulfilled  the  information  needs  required  to  meet  the 
management  function  of  controlling  the  organization.  The 
section  under  study  has  no  performance  improvement  system, 
only  a  management  control  system.  Although  the  two  are  very 
similar  in  appearance,  they  are  different  in  function. 
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IV •  Conclusiona  and  Recommendations 

Measurement  System 

The  measurement  system  used  by  the  design  section  at 
Tinker  AFB  meets  the  basic  requirements  necessary  to  be  a 
performance  measurement  system.  However,  it  lacks  the 
precision  required  to  differentiate  between  small 
differences  in  performance.  The  reason  for  this  appears  to 
be  a  difference  in  purpose.  The  researcher  was  expecting  a 
performance  measurement  system;  instead,  the  purpose  of  the 
system  seems  to  be  management  control.  The  similarities 
between  the  two  types  of  systems  exist  because  both  serve 
one  of  the  basic  managerial  functions  of  control ,  only  the 
former  adds  to  that  function  the  ability  to  closely  measure 
performance  with  the  expressed  purpose  of  improvement. 

The  performance  improvement  aspect  of  the  system  was 
almost  entirely  missing.  The  main  reasons  for  this  seem  to 
be  a  perceived  lack  of  importance  and  ability  to  change 
regulatory  hindrances.  The  researcher  found  no  working 
knowledge  of  the  executive  order  that  directs  productivity 
improvement.  The  Air  Force  command  structure  may  have 
knowledge  of  this  executive  order,  and  may  have  indeed  done 
something  about  it,  but  there  is  no  evidence  of  this 
knowledge  at  the  squadron  level.  Additionally  there  appears 
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to  be  little  perceived  benefit  from  productivity 
improvement . 

In  the  literature  review,  it  was  shown  that  the  two 
major  sources  of  improvement  in  performance  came  from 
changing  rules  that  hinder  the  worker  and  eliminating 
unnecessary  work.  The  perception  of  those  interviewed 
seemed  to  be  that  they  could  not  change  either  the  work 
they  were  required  to  do,  or  the  rules  governing  them. 

Seldom  can  a  squadron  level  section  chief  decide  that 
something  is  unimportant  and  should  not  be  done.  Instead, 
what  the  design  chief  had  done  was  shift  the  emphasis  so 
that  design  was  given  more  emphasis  than  other,  less 
important,  work.  Most  of  the  measures  taken  were  of  the 
design  function.  The  only  way  of  monitoring  other  work  was 
with  a  suspense  slip.  The  suspense  slip  served  only  to 
assure  timeliness  and  completion,  both  of  which  are 
necessary  for  effectiveness.  The  chief  of  design  at  Tinker 
AFB  seemed  to  have  little  interest  in  trying  to  change 
rules  and  regulations  that  hinder  the  performance  of  the 
section.  One  reason  for  this  is  shown  in  the  literature 
review.  In  the  literature  review  it  was  shown  that  one  of 
the  side  benefits  to  a  structured  approach  at  developing  a 
performance  measurement  system  was  identifying  regulatory 
hindrances.  Not  having  gone  through  such  an  approach,  it 
is  reasonable  to  expect  no  such  outcome  would  develop. 
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Simply  enough,  the  hindrances  are  part  of  the 
organizational  structure,  and  generally  have  been  there 
long  enough  to  be  ignored  as  such. 

The  models  used  by  past  researchers  tried  to 
incorporate  most  of  the  other  work  done  by  the  design 
section  (suggestions,  work  orders,  reports,  and  Prime  Base 
Emergency  Engineering  Force  requirements  among  other 
things)  into  the  measurement  system.  This  design  section 
simply  strived  to  assure  other  work  was  finished  in  a 
timely  manner.  This  method  seems  to  work  well  for  the 
design  section  at  Tinker  AFB . 

One  possible  remedy  to  the  perception  of  not  being 
able  to  change  the  system  would  be  the  implementation  of 
the  model  installation  program  (MIP) .  The  researcher  has 
seen  how  MIP  can  change  this  perception  rather  quickly.  In 
the  researcher’s  opinion  MIP  still  lacks  the  formal  method 
of  identifying  problems.  Instead,  it  takes  on  the 
character  of  what  would  normally  be  follow-up.  One  at  a 
time  as  workers  encounter  problems,  those  problems  surface 
in  the  MIP  system  and  are  remedied. 

In  general ,  the  measurement  system  developed  by  Tinker 
AFB ’ s  design  section  is  similar  to  what  the  literature 
review  recommends.  It  incorporates  the  same  measures: 
effectiveness,  performance,  efficiency,  and  some  measure  of 
quality.  The  system  is  also  simple  and  understandable. 
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However,  the  system  lacks  the  ability  to  quantify  several 
of  the  performance  measures. 


Comparisons 

Comparing  the  performance  measurement  system  with  the 
information  provided  in  the  literature  review  shows  both 
similarities  and  differences.  The  similarities  are  in  the 
things  being  measured.  The  system  observed  does  measure 
each  of  the  areas  suggested  as  most  important  in  the 
literature  review:  effectiveness,  productivity, 
efficiency,  and  quality.  The  basic  structure  is  provided  in 
that  the  most  important  areas  of  performance  are  measured. 

Ef f ectiveness  was  stressed  as  being  most  important  to 
engineering  performance  measurement.  The  section  under 
study  has  a  method  of  determining  effectiveness  that  works 
well,  and  satisfies  management. 

There  are  differences  in  the  system  and  the  literature 
review.  The  differences  nearly  all  stem  from  a  single 
source:  purpose.  The  purpose  of  the  system  under 
investigation  was  to  provide  the  information  necessary  for 
management  to  control  the  organization,  more  specifically 
to  meet  the  design  schedule.  The  purpose  in  the  literature 
review  was  always  to  improve  performance,  and  always 
started  with  a  perceived  need  to  improve  performance. 

During  the  research,  no  such  perceived  need  was  observed. 
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It  is  this  perceived  difference  in  purpose  that  caused 
the  most  noticeable  lack  in  the  system  under  study. 

Because  the  emphasis  was  on  maintenance  rather  than 
improvement  of  performance,  there  were  no  devices  in  place 
to  improve  performance.  No  management  action  teams  or 
study  groups  were  in  place.  The  only  real  effort  to  improve 
performance  was  in  improving  equipment  and  quality  of  the 
office  space,  and  sending  the  engineers  to  schools  and 
seminars  to  improve  their  skills.  These  schools,  especially 
AFIT  short  courses,  have  been  very  effective  in  improving 
the  efficiency  of  the  engineers  (16). 

The  second  perceived  weakness  in  the  system  they  use 
may  just  be  one  of  preference.  This  researcher  would 
prefer  a  more  quantifiable  system,  to  allow  more  precise 
measurement,  which  would  allow  management  to  more 
accurately  assess  small  changes  in  performance.  The  system 
under  use  in  the  section  here  really  is  not  quantifiable, 
but  instead  provides  bounds  that  the  section  can  monitor. 
Although  this  would  not  lend  itself  well  to  fine  tuning 
performance,  it  does  adequately  fill  management's  present 
needs . 

General  Conclusions 

This  research  lead  down  many  dark  alleys,  and  a  great 
many  things  were  learned  in  areas  that  were  not  intended  to 
be  studied.  These  extraneous  lessons  fall  into  three  broad 


62 


areas.  First,  there  is  a  serious  lack  of  communication  in 
the  United  States  Air  Force  about  performance  and 
productivity.  Second,  most  of  the  past  research  focus  on 
measurement  with  little  emphasis  on  actually  improving 
performance.  A  simple  model  such  as  the  one  used  at  Tinker 
AFB  may  provide  the  accuracy  necessary  to  start  working  on 
improvement.  And  third,  there  is  little  incentive  to 
improve  performance,  and  often  the  opposite  exists,  which 
is  motivation  not  to  improve. 

The  researcher  was  first  confronted  in  the  literature 
review  with  the  lack  of  common  definitions  for  performance 
and  productivity.  The  two  terms  are  often  used 
interchangeably.  In  the  Air  Force,  these  terms  are  Just  as 
confused.  The  researcher  found  several  times  at  the 
beginning  of  this  research  that  conversations  were  nearly 
meaningless  because  of  this  lack  of  common  definition. 

To  perpetuate  this  lack  of  common  definition,  there  is 
little  communication  about  the  subject  in  the  Air  Force. 

The  various  sources  of  continuing  education  offered  do  not 
include  the  study  of  performance  improvement  or 
measurement.  There  is  also  no  easy  way  for  one  squadron  to 
pass  on  information  to  the  rest  of  the  Air  Force.  Part  of 
the  reason  for  this  is  that  there  is  not  any  publication 
specifically  for  Air  Force  Civil  Engineering  at  this  time. 
All  of  this  lack  of  communication  causes  the  next  problem. 
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There  is  not  any  working  knowledge  of  available  help. 
The  only  resources  known  to  any  of  the  people  who  were 
interviewed,  are  AFIT  short  courses  and  various  management 
classes  available  locally.  No  one  had  heard  of  MGEEM,  and 
knowledge  of  any  other  performance  Improvement  system  was 
very  limited. 

The  second  area  of  unexpected  learning  came  mostly 
from  the  literature  review.  Sheehy  said  that  the  higher  in 
the  structure  you  go,  the  greater  the  affect  of  change 
(27:144)  His  observation  is  unquestionable,  yet  there  is  a 
second  similar  observation.  The  farther  down  the  chain  of 
command  one  goes,  the  less  the  perception  of  ability  to 
implement  change.  This  has  been  partially  overcome  by 
programs  like  MIP,  but  only  in  specific  locations. 
Unfortunately,  it  appears  that  most  of  the  Air  Force 
research  in  the  past  has  been  at  the  squadron  level ,  which 
is  where  the  ability  to  implement  change  appears  to  be 
lowest . 

A  confounding  factor  in  the  literature  is  that  most  of 
it  is  about  business.  The  literature  about  the  public 
sector  is  usually  of  cities  or  small  federal  government 
agencies.  With  these  organizations  there  are  not  the  many 
layers  of  command  that  occur  between  the  squadron  level  of 
the  Air  Force  and  top  management.  Some  of  that  top 
management  is  completely  out  of  the  Air  Force,  such  as 
Congress,  which  often  does  not  share  the  same  goals  and 
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concerns.  If  real  performance  improvement  is  to  come  to 
the  Air  Force,  it  must  come  from  top  management.  This 
includes  all  of  the  major  commands  and  the  Air  Force 
Engineering  and  Services  Center,  as  well  as  necessary 
congressional  support.  It  appears  at  present  that  some 
interest  exists  at  the  higher  levels,  but  it  is  badly 
diluted  by  the  time  it  gets  to  the  squadron  level. 

One  of  the  things  the  researcher  observed  was  the  way 
major  command  performance  improvement  programs  work. 

During  the  case  study,  the  researcher  was  told  of  a  new 
program  from  the  major  command  for  quality  improvement. 

The  perception  of  the  program  was  that  it  was  imposed  and 
would  be  of  little  or  no  help.  This  was  followed  by  the 
perception  that  this,  like  other  similar  programs,  would  go 
away  with  the  next  change  of  command  ceremony  (14).  Under 
these  circumstances,  even  a  good  program  is  likely  to  fail. 

Finally,  there  is  little  perceived  need  to  improve 
performance,  and  probably  less  perceived  ability.  However, 
there  is  a  perceived  need  to  measure  and  control 
performance,  and  that  opens  a  door  for  future  improvement. 
In  the  researcher’s  opinion,  part  of  the  reason  for  this 
perceived  lack  of  need  is  that  there  is  little  incentive  to 
improve  performance.  There  is  a  common  perception  that 
performance  improvement  can  lead  to  decreased  manpower  and 
budgets  (19:152).  Capital  improvements  appear  especially 
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prone  to  causing  manpower  losses.  This  causes  some  managers 
to  look  at  real  attempts  at  performance  improvement  as 
risky.  The  researcher  saw  another  problem  faced  early  in 
the  MIP  program.  The  problem  was  that  of  who  gets  the 
dollars  saved.  If  the  organization  that  is  saving  the  money 
gets  to  use  even  a  portion  of  that  money  for  other  things, 
it  provides  incentive  to  improve.  This  ability  to  reprogram 
saving  is  not  usually  the  case  outside  of  the  MIP  program. 

The  personnel  system  may  also  act  as  a  hindrance, 
because  of  the  excessive  amount  of  control  faced  by 
management.  In  the  literature  review,  it  was  shown  that 
first  line  managers  should  have  the  authority  to  hire  and 
fir?  employees.  Management  does  not  always  perceive  this  to 
be  the  case.  Even  though  hiring  is  supposedly  a  management 
function,  management  is  told  when  they  can  and  cannot  hire, 
based  on  strict  manpower  standards,  and  the  applicants  are 
screened.  Additionally,  the  researcher  does  not  believe 
that  managers  perceive  a  real  ability  to  fire  an  employee. 
Many  managers  also  complain  about  how  restricted  they  are 
in  the  ability  to  use  pay  as  an  Incentive.  Mot  only  can 
they  not  give  bonuses,  but  they  cannot  give  pay  increases 
based  on  performance,  except  in  very  few  circumstances  (7). 

Areas  For  Further  Research 

One  of  the  problems  with  this  system  was  with  the 
efficiency  measure  which  used  an  estimate  of  the  design 
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time  needed  to  complete  a  project.  A  program  has  been 
developed  to  estimate  design  time  and  has  been  used  at 
Keesler  AFB .  This  program  needs  to  be  examined  and  its 
accuracy  verified.  The  addition  of  the  estimating  program 
to  Tinker  AFB ’ s  existing  design  schedule  program  would  be 
very  valuable.  A  more  valuable  area  for  further  research 
would  be  to  focus  on  performance  improvement  over 
performance  measurement. 
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ABSTRACT 


The  design  section  is  responsible  for  designing 
projects  for  completion  by  contract,  overseeing  designs 
done  by  architect-engineer  firms  or  the  Army  Corps  of 
Engineers  (COE) .  -^The  impact  a  design  section  has  on  an  Air 
Force  base  is  enormous.  The  programmed  budget  for 
construction  for  1987  was  over  $3  billion,  or  about  15 
percent  of  the  Air  Force  operations  and  maintenance  budget. 


"'-Performance  is  important  in  any  organization  with  this 
large  of  an  impact.  That  performance  should  constantly  be 
improved.  Before  performance  can  be  improved  it  must  be 
accurately  measured.  The  literature  available  suggests 
that  to  accurately  measure  the  performance  of  engineers, 
one  must  first  identify  key  dimensions  of  performance 
which,  when  accomplished,  will  assure  effectiveness.  After 
effectiveness  has  been  assured,  then  efficiency,  produc¬ 
tivity  and  quality  need  to  be  measured.  Together  these 
four  things  give  a  picture  of  performance. - ^  K  • .  • 


A  case  study  of  an  Air  Force  Civil  Engineering  design 
section  was  done  with  the  purpose  of  documenting  the  system 
used  for  performance  measurement.  The  system  used  measured 
effectiveness,  performance  against  schedule,  efficiency 
and  productivity.  Some  capability  of  assuring  quality 
was  also  built  into  the  management  system. 


The  system  that  is  used  by  the  section  under  study 
does  provide  a  limited  capability  to  measure  performance 
However,  this  system  uses  mostly  subjective  measures  and 
little  actual  quantifiable  data.  This  is  a  weakness  if 
fine  differences  in  performance  need  to  be  measured,  as 
would  be  the  case  for  a  performance  improvement  system. 
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